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Abstract 

Corporate and educational settings increasingly require decision-making , problem-solving and other 
complex cognitive skills to handle ill-structured, or heuristic , tasks , but the growing need for heuristic task expertise 
has outpaced the refinement of task analysis methods for heuristic expertise. The Heuristic Task Analysis Method 
was applied to three settings to generate improvements and more detailed guidance , and to identify variations in the 
method for different situations. The three settings were group counseling, tutoring on writing skills , and selecting 
artwork for a product line. The formative research methodology was used to test the method and generate 
improvements. The three studies produced some common and some unique findings and recommendations. A 
tentative revision to the HTA method is proposed. 



Introduction 

As our society in general and the workplace in particular become more complex, we are finding that a 
greater number of the activities that people undertake are relatively more heuristic in nature than ever before. 
Whether in K-12 education, higher education, corporate training, or any other context, to help people learn the 
heuristic elements of an expert's know-how, we must be able to identify those heuristics. 

In analyzing heuristics, we find it helpful to think in terms of two major kinds of expertise domain and 
task expertise. Task expertise relates to the learner becoming an expert in a specific task, such as managing a 
project, selling a product, or writing an annual plan. Domain expertise relates to the learner becoming an expert in a 
body of subject matter not tied to any specific task, such as economics, electronics, or physics (but often relevant to 
many tasks). (Reigeluth, 1999, p. 435). 

Both procedural and declarative knowledge are important elements of both kinds of expertise. In this research, we 
focus on task expertise. For task expertise, we find it helpful to think in terms of two major kinds of tasks; 
procedural and heuristic. Procedural tasks are "tasks for which experts use a set of steps, mental and/or physical, to 
decide what to do when, such as a high school course on mathematics or a corporate training program on installing a 
piece of equipment for a customer. Heuristic tasks are "tasks for which experts use causal models— interrelated sets 
of principles and/or guidelines — to decide what to do when, such as a high school course omthinking skills or a 
corporate training program on management skills." (Reigeluth, 1999, p. 435). 

The distinction between procedural and heuristic tasks is similar to the distinction between well structured 
and ill structured domains (Fredericksen, 1984; Resnick, 1988; Simon, 1973). In reality most tasks are neither 
purely procedural nor purely heuristic, but some combination of the two. We have relatively powerful 
methodologies for analyzing the expertise that underlies procedural tasks (i.e. the mental and physical steps upon 
which an expert relies). But we do not have good methodologies for analyzing the expertise that underlies heuristic 
tasks. This situation is exacerbated by the reality that heuristic knowledge is frequently tacit rather than explicit — 
that is, experts are often unaware of the heuristics that guide their performance. Therefore, there is a strong need to 
develop task analysis methodologies for identifying the knowledge that underlies heuristic tasks. The full paper 
reviews literature related to heuristic task analysis. Then it describes three research studies that have been 
conducted to improve one of those methodologies. This paper is a summary of the full paper. 

The Heuristic Task Analysis Method of Elaboration Theory 

The Elaboration Theory (Reigeluth, 1999; in press) offers the Heuristic Task Analysis (HTA) method as 
part of its Simplifying Conditions Method (SCM) for task analysis. This more general method of task analysis is 
conducted by asking the question, "What is the simplest version of the task that an expert has ever performed?" and 
"What is the next simplest version?" and so forth. As each version is identified, its place in the learning sequence is 
simultaneously determined. Therefore, the SCM task analysis method is an integral part of the method for designing 
an instructional sequence. Furthermore, since most tasks have a combination of procedural and heuristic elements, 
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the procedural and heuristic task analysis methods are integrated into a single process. For these reasons and 
because this method offers a fair amount of guidance for the task analysis process, we chose this method for our 
research. Hence, it is described in greater detail next. 

The SCIVTs Heuristic Task Analysis Method 

The following are some details on the SCM's heuristic task analysis method. They are an elaboration of the 
process described by Reigeluth (1999). 

Phase I. Prepare for Analysis and Design 

1. Prepare. Lay the groundwork for your analysis and design. 

1 . 1 Establish rapport with a task expert. 

1 .2 Explain the analysis process you will be using. 

1 .3 Ask the task expert about the nature of the task in general. 

1 .4 Identify the characteristics of the learners in general. 

1 .5 Identify the delivery constraints of the task in general. 

Phase II. Identify the First Learning Episode 

2. Identify the simplest version. Help the task expert to identify the simplest version of the task that is fairly 
representative of the task as a whole, and to describe the conditions that distinguish that version from all other 
versions. 

1. You may want to use some other criteria in addition to simple and representative, such as common and 
safe 

2. Ask the task expert to recall the simplest case she or he has ever seen. The simplest version will be a class 
of similar cases. Then check to see how representative it is of the task as a whole. 

3. It may be helpful to start by identifying some of the major versions of the task and the conditions that 
distinguish when one version is appropriate versus another. 

4. Thinking of different conditions helps to identify versions, and thinking of different versions helps to 
identify conditions. Hence, it is wise to do both simultaneously (or alternately). 

5. There is no single right version to choose as the "simplest.” It is usually a matter of trade-offs. The very 
simplest version of the task is usually not very representative of the task as a whole. The more 
representative the simple version can be, the better, for it provides a more useful schema to which learners 
can relate subsequent versions. 

6. It may be wise to go through this process with several task experts before going on to Step 3. You may 
find it necessary to take steps to resolve differences of opinion about which is the best “simplest version” to 
use. 

3. Analyze the organizing content. Analyze the organizing content for this version of the task. 

3.1 Ask the task expert to think of one specific performance of the task to analyze, or to videotape a 
performance for you to review with the expert during the analysis. 

3.2 Use a top-down approach to analyzing the content (the knowledge upon which the expert’s performance is 
based). In other words, start by identifying the general categories of knowledge that an expert uses, then 
proceed to analyze each. 

7. Ask the task expert: 

- to describe each decision that the task expert made, 

- to identify the kinds of knowledge that the task expert drew upon to make the decision, and 

- to describe the specific knowledge within each kind of knowledge that the task expert used. 

8. The kinds of knowledge are likely to include: 

- steps (procedural knowledge), 

- guidelines or rules of thumb (heuristic knowledge), 

- explanatory models (which explain why the guidelines work), 

- descriptive models (which describe the phenomena with which the task expert interacts), and 

- metacognitive/decision rules (which the task expert uses to decide which steps, guidelines, and 
descriptive knowledge, to use when). 

9. It is generally helpful to start by asking the task expert if there are any steps or phases of activities that 
are always performed for this version of the task. If so, perform a procedural task analysis to identify 
the sequence of steps and to see if any of those steps can be broken down into substeps, but those 
substeps must be ones that an expert thinks of and uses routinely in performing that version of the task. 

10. For guidelines, use the following process: 

1 . Identify the goals for this specific performance of the task under its conditions. 
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2. Identify all the important considerations for attaining each goal. Considerations are the major 
categories of causal factors that influence performance of the task. If there are a lot of causal factors 
for a consideration, it is useful to identify subconsiderations for it. 

3. Identify all the important causal factors for each consideration (or subconsideration). 

4. Analyze each causal factor to identify all guidelines (prescriptive principles or “rules of thumb ) that 
an expert uses to account for this consideration. 

1 1 .For explanatory models, use the following process: 

12. For each guideline, ask the task expert for the reasons why s/he believes it works. 

13. For interrelated guidelines, you are likely to identify a set of related reasons that constitute a causal 
model or models. Be sure to look for multiple causes for each effect and multiple effects for each cause. 
Also look for chains of causes and effects, and explore probabilities for each causal factor to have each 
effect. 

14. For descriptive models, use the following process: 

1. Ask the task expert what phenomena influenced this particular performance of the task. Try to 
identify all causal relationships that characterized those phenomena. Be sure to look for multiple 
causes for each effect and multiple effects for each cause. Also look for chains of causes and effects, 
and explore probabilities for each causal factor to have each effect. 

1 5. For metacogniti ve/decision rules, use the following process: 

1. Find out what rules the task expert used to decide when to use which steps, guidelines, and 
descriptive models during the specific performance of the task being analyzed. 

16. It is wise to query the task expert about any of these kinds of knowledge that are not initially described 
to you for each decision the task expert made in this specific performance of the task. 

3.3 Ask the task expert to think of similar performances of the task that constitute a single version of the task. 
Use each such performance to broaden the steps, guidelines, explanatory models, descriptive models, and 
metacognitive/decision rules so that they represent the knowledge bases the task expert uses to deal with all 
performances for that version of the task. 

3.4 If time and resources permit, find a second task expert with whom to repeat this entire process (Steps 1— 
3.3) to identify any alternative views of the task and the knowledge that underlies its performance. It may 
even be wise to repeat this process with several more task experts. And you may want to go back and ask 
each task expert what s/he thinks about the perspectives of the other task experts, in an effort to reconcile 
conflicts and select among alternative ways of thinking about and performing the task. 

The HTA method has not been rigorously tested and therefore is in need of further research. However, the 
most important research issue is not the validity of the HTA method, for, like most methods, it is likely to work 
some times but not always, and to varying degrees. Rather, given the immaturity of our knowledge about how to 
analyze heuristic tasks, what is needed most at this point is developmental research— research that is intended to 
further develop and improve the method. Therefore, our research question is, M In what ways can the HTA method 
possibly be improved?" To answer this question, it is also necessary to find out what parts of the method are 
working well and what parts are not working so well. Furthermore, to improve the HTA method, it will likely be 
important (1) to change parts of the method, (2) to provide more detailed guidance about how to accomplish 
particular parts of the method, and (3) to identify variations in the method for different situations, such as different 
kinds of tasks or even different kinds of task experts. 

To answer these research questions, we conducted a series of three developmental research studies. These 
are described next, followed by some general conclusions. 

Study 1: The HTA Method Applied to Group Counseling 

The purpose of this study was to improve the HTA method and guidance for use of that method. Thus, the 
formative research methodology was adopted with emphasis on exploring how the HTA method can be improved 
when applied to group counseling. 

Formative research is a kind of developmental research or action research that is intended to improve 
design theory (Reigeluth & Frick, 1999). In contrast to research on descriptive theory, which emphasizes validity or 
how well the description matches the reality of "what is," research on design theory is more concerned with 
preferability, the extent to which one method is better than other methods for achieving certain goals under certain 
circumstances. By creating or identifying an instance of a design theory and collecting formative data to improve 
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that instance, one may develop better understanding of how the theory works in the field and thus be able to propose 
improvements for the theory, which of course would need to undergo further testing. 

Task. Group counseling was selected as the task to which to apply the HTA method. Group counseling is 
a combination task with both procedural and heuristic elements. It is procedural in that the activities of the group 
leader are largely determined by the stages that a group goes through (i.e., forming, norming, storming, and 
performing), and the leader cannot help the group to progress to the next stage without performing certain tasks 
(steps) at each stage. However, at a deeper level of analysis, the knowledge required for the leader to decide when 
and how to intervene is not a set of steps but a set of guidelines and principles, which are heuristic knowledge. 

Participants. This study involved three participants as task experts in group counseling. Expert #1 was a 
professor in the Counseling Department in the Indiana University School of Education and was the most 
experienced of the three task experts. The other two were doctoral students in the same department. They were all 
experienced in conducting "personal growth" group counseling, and their expertise ranged from three to more than 
20 years. 

Data Collection Methods 

Interview. Semi -structured in-person interviews were used as the primary data collection method. The 
second author conducted six interviews between September and November, 1999. Each interview took 30-90 
minutes and was audio taped for analysis. The purpose of the interviews was to find ways to improve the HTA 
method for eliciting, analyzing, and representing the expert's heuristic knowledge for performing the task of 
"personal growth" group counseling. The investigator played two roles, one as a task analyst proficient in the HTA 
method and the other as researcher searching for ways to improve the HTA method. As task analyst, the 
investigator developed a set of interview questions (see Appendix A) for the interview based on the HTA method, 
but as researcher the investigator was not restricted to the predefined questions. Depending on the expert's response, 
the researcher revised the HTA method for the next interview. Thus, the overall interview process was flexible and 
reflective in nature. 

Videotapes. Because of the confidential nature of group counseling, direct observation or videotaping of 
an expert's task performance (as called for by the HTA method) was not allowed. Instead, the researcher (as 
analyst) used a series of instructional videos that simulated group counseling sessions for beginning group leaders, 
to provide the analysts with a concrete case. 

Data Analysis and Interpretation Methods 

The HTA method is an iterative process: finishing the first round of HTA is not the end of the study but the 
beginning of the second round of HTA; and the end of the second round is, again, the beginning of the third round; 
and so on. The investigator went through two rounds of HTA in this study. 

Triangulation. To enhance the thoroughness of the data, this study involved three experts as data sources. 
Each of them played somewhat different roles during the interviews. During each round of data collection, expert 
#2 provided the initial structure of the task setting and knowledge base. Then experts #1 and #3 reviewed the 
knowledge elements, verified them, and provided additional information. There were a few times when the three 
experts did not agree with one another. In such cases, expert #1 *s judgment was accepted, as he was the most 
experienced group counselor. 

Member checks. After each interview with an expert, the researcher transcribed the interview and took the 
summary and interpretations to the next interview for review. Through this process, the experts corrected errors or 
misconceptions by the researcher, and the researcher asked additional questions to clarify the information. 

Consultation. During the data collection and analysis process, the researcher regularly met with the other 
three researchers in this study and consulted them in designing the interview protocol and analyzing the data. 

Results and Discussion 

The first round of data collection involved initial interviews with three experts. Instead of finishing with 
one expert and then starting with another, the researcher worked with the three experts simultaneously (but 
separately) due to their time schedules. This approach involved some tradeoffs. It worked well in the sense that the 
researcher could get the three experts to reach consensus on the simplest version of the task early in the HTA 
process. However, communicating with all three experts simultaneously was not easy for the researcher, and the 
researcher had to spend most of the time during the interviews explaining to each expert the previous interviews 
with other experts. Even though the task was a common one, the experts still had difficulty explaining the detailed 
decision-making process when the task had originally been defined by another expert. Facing this problem, the 
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researcher decided to use an existing instructional video series (with which all three experts were familiar) as a 
frame of reference, instead of trying to build a new scenario based on each expert’s experience. 

The second problem was that the researcher lacked expertise in the task of group counseling. The 
researcher found that, to be able to push the expert to further elaborate his/her automatized (and hence subconscious) 
task expertise, the analyst needed to speak the same language as the expert and be able to prompt when the expert 
had difficulty in finding the right words. Without a certain level of expertise in the task, the researcher as task 
analyst had difficulty doing those jobs smoothly. 

The third problem was related to the difficulty of categorizing the types of knowledge underlying each 
decision made by the expert during the task analysis. The purpose of identifying the five types of knowledge 
identified by the HTA was to make sure that the expert did not overlook one of the important types of knowledge, 
but the benefits of distinguishing among the types seemed to not be worth the extra time required in this case. 

The second round of HTA incorporated some new methods to deal with the problems found in the first 
round. First, the researcher summarized key incidents from the video series on index cards and used them as a 
reference during the interviews with the experts. This was very helpful in three ways: (1) it helped the experts to 
recall details about the task performance process, (2) it helped both analyst and experts to see the flow of the task 
performance process and get back on track when the experts got off-task, and (3) it saved a lot of time in revisiting 
previous points. One expert commented that the index cards forced him to be more precise during the review and 
revision process. Second, the researcher as analyst used a bottom-up approach (identifying knowledge first, then 
categorizing it as to type) rather than the top-down approach (identifying knowledge within each type) suggested by 
the HTA method. 

Based on the findings of this study, the following changes are proposed as possible improvements and 
described in detail below: (1) incorporate various interview and observation techniques into the HTA process, (2) 
provide different guidelines for analysts with different levels of task expertise, (3) provide different guidelines for 
working with task experts with different levels of expertise, and (4) provide reference material during the interview 
with task experts. 

Study 2 - The HTA Method Applied to Tutoring on Writing Skills 

As in Study 1, the purpose of this study was to improve Reigeluth’s HTA process by using the formative 
research methodology. This study followed the steps suggested by Reigeluth and Frick outlined in study 1. 

Task. The heuristic task chosen for this study was tutoring university undergraduate students who needed 
extra assistance with their writing skills. Specifically, the task concerned the decision-making process in which an 
expert writing tutor engages to determine the direction and focus of the tutoring session. By its very nature, a 
tutoring session requires a lot of heuristic expertise, because it is determined more by events that occur during the 
tutoring event than by any predetermined procedural steps. What occurs during the tutoring session depends on both 
the writing situation and the tutee. The writing situation includes why the tutee is being tutored, the relationship 
between the tutee and the teacher, the interest level of the topic being written about, and the number of drafts already 
written. The tutee includes any previous experiences, both positive and negative, that the tutee brings to the tutoring 
session. 

Participants. Two experts were chosen based on their level of expertise and the approval of their 
supervisors. Both experts had extensive experience tutoring all levels of writing students, and both were highly 
recommended by their writing center supervisors. A third tutor was also recommended and interviewed as a 
potential participant in this study. However, the recommendation came without the experience and evaluation 
credentials listed above, so he was not included in this study. 

Data Collection Methods 

Interview: As in study 1, the main data gathering method was the personal interview, and the researcher 
both elicited heuristic knowledge (analyst role) and conducted formative research (researcher role). Two interviews 
were conducted within one week of the tutoring session that was being analyzed. Both of the interviews were 
conducted within one week of the tutoring session that was being analyzed. Because of the tutors’ lack of time to 
spend on this research, each interview was limited to approximately 60 minutes. Prior to the interviews, each of the 
tutors was sent emails describing terms used, definitions, an outline of the interview questions, and a brief 
explanation, written in their terms, of the purpose, expected results, and use of this research. 

Before the actual interview, the researcher reminded each tutor of what was sent to them earlier and asked if any 
terms or points needed to be clarified. At this time, the researcher also pointed out to each expert writing tutor that 
a) it was unclear whether the task about to be analyzed was actually based on heuristic knowledge and b) it was 
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unclear whether the questions would be able to access that knowledge. This was done to reduce any anxiety that the 
writing tutors might experience if they could not produce information that the researcher desired. 

Based on the first interview, the HTA methodology was altered slightly for the second interview so as to 
assist the tutor to better recall the tutoring situation. The analyst had the tutor respond to specific questions about the 
actual tutoring experience, the tutee’s characteristics, and the tutee’s essay prior to having the tutor recall and reflect 
on her decision-making processes. He then had the tutor identify the decision areas to focus on during the tutoring 
session. Afterwards, he had the tutor choose the concern that was most available to her. This, then, became the 
subject of the heuristic task analysis. 

Data Analysis and Interpretation Methods 

There were no follow-up interviews or member checks as in Study 1 to determine the validity of the tutors 
responses due to the writing tutors’ lack of time to spend on these tasks. However, because of the researcher s 
expertise in this area, he concluded that the data collected was not spurious. At the conclusion of each interview, the 
experts were asked to review and modify what was recorded during the interview. The researcher asked each expert 
for ways to improve the interview process and to comment on its effectiveness in eliciting the knowledge underlying 
their decision-making thought process. Both offered suggestions about ways to help them recall the previous 
tutoring situation and about the limitation of focusing on only one aspect of the tutoring process. The second expert 
confirmed what the first had concluded. After each interview, tentative changes were made to the HTA process. 

Results and Discussion 

In the interview after the first application of the HTA method, the first tutor mentioned how the process 
helped him think about his own tutoring strategies. He also mentioned that having to recall from memory a tutoring 
session that was done even within the last seven days was not easy. The tutor suggested the following refinements 
to the HTA method. 1) The top-down process seemed effective. 2) Because the tutor experienced some difficulty 
recalling the specific tutoring session, the analyst (researcher) asked some specific questions about the tutee, the 
paper, and the tutee’s reactions to the tutor’s suggestions. Both tutors said this helped them get into the flow of the 
previous tutoring session, and the researcher observed a marked increase in awareness and confidence after assisting 
the tutor’s recall. 3) 5x8 cards were effective in that the tutor referred back to them to align his insights into the 
tutoring process with previous statements. 4) The tutor, when identifying the guidelines, focused more on how to 
hold an effective tutoring session than on what influenced his decision to focus on a specific tutoring objective. In 
addition, 5) the researcher suspected that the results of the HTA might have been richer if the task expert (tutor) had 
been given more control over the decision point selection process. 

In summary, an important concern involves the task expert’s tendency, when explicating the guidelines, to 
focus on the goals and not on the decision points for attaining the goals. During both instantiations, the writing 
tutors gave the guidelines they used for deciding on the goals of the tutoring situation rather than giving guidelines 
for deciding how to attain a goal during the tutoring session. When this occurred, the researcher gently prodded the 
experts to focus on the decision points rather than the goals. However, when the experts could not provide that 
information, the researcher decided to review previous sections and then ask that question again. After the experts 
referred to the goals again, the researcher decided not to push them any further, seeing that they both were unable to 
provide that information. Another concern involves the first expert’s difficulty in recalling the tutoring session 
despite the fact that the session occurred only one week prior to the interview. Measures taken to assist the second 
expert’s ability to recall the tutoring session showed a dramatic improvement. 

Study 3: Selecting Artwork for a Commercial Product Line 

The third study tested the HTA method in a corporate setting. As in the previous two studies, formative 
research was the methodology, using a designed case to generate possible improvements in the HTA method. 
Corporate executives want a “big bang for their buck,” and analysis is often looked upon as a time-consuming 
activity with questionable impact. The aim of this study was to develop a rapid, high-impact version of the HTA 
method for corporate settings. Thus, the study was designed to provide insight into the following research questions. 
(1) How can the speed and effectiveness of the HTA method be improved for eliciting, analyzing, and representing 
heuristic knowledge from experts in corporate settings? (2) What guidance could be added to the method to assist 
analysts in corporate settings? The time constraints for this study dictated that the research be limited to a single 
interview cycle with one task expert, lasting no more than a total of three to four hours. 

Task. The heuristic task chosen for this study was deciding whether a submission of artwork was suitable 
for one of the company’s product lines. This was a judgmental decision-making task requiring a fair amount of 
experience and know-how. The task expert verified that the task was important to the company, that she was 
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considered to be an expert at the task, and that it was not easy to articulate the expertise required to perform the task. 
The heuristic nature of this task was verified by an expert in the HTA method. 

Participant. The task expert was recruited by calling a local business that had collaborated with the 
Instructional Systems Technology Department at Indiana University in the past. A manager in the design department 
enthusiastically agreed to participate in the study. The expert and the analyst discussed possible complex decision- 
making tasks over the telephone and came to an agreement on an appropriate task. 

Data Collection, Analysis, and Interpretation Methods 

Interview. The researcher/analyst conducted two audio taped, one-and-a-half-hour interviews with the 
participant (task expert) in a conference room at the expert’s place of business. The analyst/researcher referred to an 
interview sheet (described below) to ensure that he was adhering to the guidelines of the HTA method, although he 
also allowed the interview to be somewhat unstructured as seemed appropriate to gather the heuristic knowledge and 
data for improving the HTA process. After one-and-a-half hours he reached a saturation point in terms of gathering 
the essence of the task and the key heuristics and concluded he could not effectively continue the analysis without 
first going back to his office and organizing the information collected. The expert agreed to continue the interview 
the following week. The analyst/researcher logged “significant chunks” of the audio tape on 3” x 5” cards. His 
criterion for “significant chunks” was any piece of knowledge that fit into one or more of the types of knowledge 
listed in the HTA method. He examined these knowledge elements to determine what missing ones he needed to ask 
about in the follow-up interview. Then in the role of researcher, he re-examined the interview results for deviations 
from the HTA method to see where the method was effective in eliciting heuristic expertise and where deviations 
were helpful. He discussed his preliminary findings with Reigeluth and worked with him to plan the second 
interview. 

Results and Discussion 

Speed of the HTA method. The analyst/researcher found a number of areas in which the speed of the 
HTA method might be enhanced. Two are discussed in this section. The others are the result of improving the 
effectiveness of the method and are discussed in the next section. The analyst/researcher noticed that almost an hour 
was spent identifying the simplest version of the task and distinguishing it from other possible versions. This can be 
important for training purposes, as outlined in the SCM methodology. However, in a business context, there can be 
other purposes for conducting the heuristic task analysis. The results of such an analysis can be used to generate job 
aids for experts, to help designers structure knowledge-management systems, and for other purposes. If training is 
not the primary purpose, then the analyst might choose to spend less time identifying the simplest version and other 
versions (Step 2. Identify the first learning episode). Such information might still be useful for distinguishing experts 
and novices, even though sequencing course material is not of concern. In this study, the analyst/researcher 
concluded that this step could have been concluded with significantly less time (approximately 20 minutes less), 
without diminishing the quality of the results. 

Recommendation: Unless using the HTA method specifically for training purposes, perform Step 2, 
"Identify the simplest version,” only if needed to distinguish between experts and novices or as one way of helping 
the expert access tacit knowledge. As the expert examines various instances of a task in search of heuristics, it may 
be helpful to distinguish between simpler and more complicated versions. 

Effectiveness of the HTA method. The HTA method seemed to be effective in its primary function of 
eliciting heuristic knowledge from the expert. The types of knowledge delineated in Section 3.2 of the HTA Method 
were found to adequately cover the range and types of task knowledge described by the expert. The 
analyst/researcher did, however, have problems managing the two tasks of classifying the expert’s knowledge and 
directing the interview to dig deeper into the expert’s tacit knowledge. More practice with the methodology should 
alleviate this. The analyst/researcher noticed during the analysis that certain verbs used by the expert were indicators 
of tacit knowledge. Examples of these verbs are: know, like, feel, see, determine, understand , and decide. When the 
analyst/researcher asked the expert why she liked a certain piece of art, she struggled at first to find reasons, but 
eventually she isolated specific characteristics that distinguished artwork she liked from pieces that she did not find 
acceptable. 

Recommendation: Additional guidance should be developed on how to represent explicit knowledge. 
Although this analyst/researcher has only begun to research this point, such guidance could come from fields such as 
task analysis or the expert’s specific field. One area of interesting research would be collaboration between the 
expert and the analyst to develop an explicit representation for knowledge deemed critical. 

Guidelines for analysts. Throughout the two interviews, the analyst/researcher made a conscious effort to 
avoid academic jargon, and the expert seemed to rapidly understand everything the researcher was saying. In 
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moments where the analyst/researcher caught himself using a technical term, he laughed it off with the expert and 

used the moment to increase rapport. 

Suggestions for Improving the HTA Process 

The following is a tentative revision of the HTA process based on the findings of these three formative 

research studies. The changes and additions are in italics . 

Phase I. Prepare for Analysis and Design 

7. Decide on a task to analyze and be clear about the reasons for analyzing it. 

2. Make sure you have enough task knowledge to have a good command of terminology and key ideas. 

• Review basic reference materials and try to become familiar with key concepts and jargon in the field. 

• It would be better to begin by identifying the simplest version of the task , rather than trying to expand the 
analysis to the next version. 

3. Make sure you have enough knowledge about the uses of the task description. 

• If the task description will be used primarily for deciding on the content and sequence of instruction, identify 
the characteristics of the learners and the delivery constraints of the instruction in general. 

4. Arrange to interview multiple experts. 

• Identify at least 2 or 3 experts to interview. 

• Plan to complete the analysis with one expert before initiating the analysis with another. 

• Plan to interview the least experienced expert first and proceed to interview progressively more experienced 
experts in order. 

• Ask one or more of the task experts to record their performance of a very simple version of the task, and review 
the recorded material in advance of the analysis; or observe the task expert's task performance. 

5. Prepare in conjunction with the first (next) task expert. 

• Establish rapport with the task expert. 

• Introduce the HTA method to the expert . 

• Explain basic terms (i.e. guidelines, explanatory models, etc.). 

6. Prepare for the interview. 

• Prepare interview materials (i.e., index cards to summarize critical incidents during task performance). 

• Practice the HTA interview process if you are not very experienced in it. 

• Arrange the intewiew logistics (e.g., reserve a conference room where you can work without interruptions). 

Phase II. Identify the First Learning Episode 

1. Identify the simplest version. Hold a focus group interview with multiple task experts, and help them to reach 
consensus on the simplest version of the task that is fairly representative of the task as a whole. Also help them 
to describe the conditions that distinguish that version from all other versions. 

• You may want to use some other criteria in addition to simple and representative. 

• It may be helpful to have the expert briefly discuss closely related tasks and clearly distinguish between the 
main task and the related tasks during the remainder of the analysis 

• Ask the task experts to recall the simplest case they have ever seen. The simplest version will be a class of 
similar cases. Then check to see how representative it is of the task as a whole. 

• It may be helpful to start by identifying some of the major versions of the task and the conditions that 
distinguish when one version is appropriate versus another. 

• Thinking of different conditions helps to identify versions, and thinking of different versions helps to identify 
conditions. Hence, it is wise to do both simultaneously (or alternately). 

• There is no single right version to choose as the "simplest." It is usually a matter of trade-offs. 

• It is wise to go through this process with several task experts together and reach consensus before going on to 
Step 8. 

8. Analyze the organizing content. With the least experienced expert you have not yet interviewed, analyze the 
organizing content (mostly heuristics and descriptive theories) for this version of the task. 

8. 1 Review the recorded material (or any other visual aid) with the task expert. 

8.2 Ask the task expert to think of and describe one specific performance of the selected version of the 
task to focus on for your analysis, or ask if a videotaped performance would be a good case for you to 
focus on with the expert during the analysis. 
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• It is often helpful to have a videotape of a typical performance of the simplest version of the task, so 
you and the task expert can review it during the analysis process, but asking the task expert to recall 
one specific performance and keep it in mind throughout the process is a more convenient and 
inexpensive, albeit often less effective, alternative . 

• If you don't have a videotape, It may be helpful to have the expert describe contextual information 
and particulars of the specific performance, describing how the expert began the case, how it 
progressed (in sequence), how participants reacted, and how the expert dealt with any problems that 
arose . 

• It may be helpful to prioritize the problems/concerns that arose and the decisions/actions that the 
expert used to deal with them. 

8.3 Decide whether to use a top-down or bottom-up approach to analyzing the content. If top-down, use 
Step 8.4 and skip Step 8.5. If bottom-up, skip Step 8.4 and use Step 8.5. 

8.4 If top-down approach, start by identifying the general categories of knowledge that an expert uses, then 
proceed to analyze each. 

• Ask the task expert: 

a) to describe each decision that the task expert made, 

b) to identify the kinds of knowledge that the task expert drew upon to make the decision, and 

c) to describe the specific knowledge within each kind of knowledge that the task expert used. 

• The kinds of knowledge are likely to include: steps, guidelines or rules of thumb, explanatory 
models, descriptive models, and metacognitive/decision rules. 

• It is generally helpful to start by asking the task expert if there are any steps or phases of activities 
that are always performed for this version of the task. If so, perform a procedural task analysis to 
identify the sequence of steps and to see if any of those steps can be broken down into substeps, but 
those substeps must be ones that an expert thinks of and uses routinely in performing that version of 
the task. 

■ • For guidelines, use the following process: 

1 . Identify the goals for this specific performance of the task under its conditions. It may help to 
have the expert explain the goals in task-specific terms rather than in abstract terms and to think 
of the goals as ideal outcomes. 

2. Identify all the important considerations for attaining each goal. 

3. Identify all the important causal factors that relate to each consideration/subconsideration. 

4. Analyze each causal factor to identify all guidelines that an expert uses to account for this 
consideration. 

• For explanatory models, use the following guidelines: 

- For each guideline, ask the task expert for the reasons why s/he believes it works. 

- For interrelated guidelines, you are likely to identify a set of related reasons that constitute a 
causal model or models. 

• For descriptive models, use the following guidelines: 

- Ask the task expert what phenomena influenced this particular performance of the task. Try to 
identify all causal relationships that characterized those phenomena. 

- Be sure to look for multiple causes for each effect and multiple effects for each cause. Also look 
for chains of causes and effects, and explore probabilities for each causal factor to have each 
effect. 

• For metacognitive/decision rules, use the following guideline: Find out what rules the task expert 
used to decide when to use which steps, guidelines, and descriptive models during the specific 
performance of the task being analyzed. 

• It is wise to query the task expert about any of these kinds of knowledge that are not initially 
described to you for each decision the task expert made in this specific performance of the task. 

• If the expert uses words such as know, feel, see, understand, like, determine, and decide, that may be 
an indication that heuristic knowledge underlies that particular performance. 

• It is often helpful to periodically ask the expert some questions about the chosen case, to keep the 
analysis focused on the flow of that version of the task. 

• It is useful to help the expert think about ways the specific case fell short of how it should have been 
done and to have the expert offer guidelines for how this specific case should have been done. 
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• It is wise to have some kind of reference material to provide contextual information and cues and to 
help the expert be more precise. During the iterative interview process, the visual aid also helps the 
expert keep on track. 

• It may be helpful to use index cards for all of these kinds of knowledge, filling them out with the task 
expert during the analysis process with one piece of knowledge per card, and arrange the cards in 
some order on a table in front of both of you, so you can easily switch from one part or aspect of the 
task to another. 

8.5 If bottom-up approach, ask the expert to describe each decision that s/he made and the process through 
which s/he went to make each decision. 

• After the interview, try to categorize each piece of heuristic knowledge according to these categories: Steps, 
guidelines or rules of thumb, explanatory models, descriptive models, metacognitive/decision rules. 

• Be sure to "member check" the interview results with the expert in a later interview to verify/ identify the types of 
knowledge underlying each decision. 

8.6 Ask the task expert to think of similar performances of the task that are within the realm of the version 
of the task you are currently analyzing. 

8.7 Repeat this entire process (Steps 5 - 8.6) with the next least experienced task expert to identify any 
alternative views of the task and the knowledge that underlies its performance. 

• For each more experienced expert, you should summarize the previous description of the task and ask the expert 

to review it, in an effort to reconcile conflicts and select among alternative ways of thinking about and 

performing the task. 

Formative research data indicate that this revised HTA process would have been more effective for the 

three cases investigated here. It remains to be seen whether or not this revised process will also work well for 

analyzing other tasks that have heuristic elements. The data in this study indicate that much additional guidance is 
still needed for conducting a heuristic task analysis. It is our hope that this study will encourage others to conduct 
additional research to improve the available guidance for analyzing heuristic tasks. 
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